Valid Comparisons 



Any conclusion drawn from an invalid premise is itself invalid 

Principle of formal logic 

In order for any measures of performance to be meaningful, certain 
conditions should be met 

Only valid, arm's length transactions that are good indicators of the value of similar 
properties should be used. Types of sales that should not be considered are: 

• Blanket Transactions 

• Speculator purchases 

• Sales between related parties 

• Partial interests 

• Bids, foreclosures or sales with no, or atypical, financing 

• Sales that anticipate a change in use of the property 

• Sales where the property itself was significantly different at time of sale than at 
time of valuation 

• Sales of extremely unusual properties 

• Sales reported from Records Department with interpolated prices 

Any study that uses unfiltered sales data is not reliable! 



Sales Price ^fa Value 



It is important to understand that Sale Price is not the same as Value. There is an 
inherent degree of variability in sales price for which no predictive model or 
technique can account. Value is an estimate of the most probable price that a 
buyer will offer and for which a seller will sell. 

Variance in price can occur for any or all of these reasons: 

• Change to the property itself (Addition, Rehab, teardown, change in use) 

• Change in market bias or desirability 

• Degree of error in time adjustment calculations 

• Short term fluctuations in supply and demand relationships 

• Attributes not present in the model (Interior condition, baths, external 
influences, traffic) 

• Relative negotiating skills of the buyer and seller 

• Financial incentives or programs (Abatements, grants, interest rates) 

• Availability of financing 

• Butterflies in China (for those who ascribe to mathematical chaos theory) 

Like horseshoes and hand grenades, getting close is what counts in the assessment industry! 



Single Family by Zone 

Using the correct source for sales data and scrubbing the file to remove invalid sales, this is the 
result of the AVI ratio study for single family residences 

Performance was measured at 3 points in time - at the start of the project; after the model 
projections; and after the Evaluation staff had reviewed the projections and made corrections 











Actual Value Initiative 


- Ratio Statistics by Zone 












Com par 


ng the AVI v 


alue of properties that sold against time adjusted prices. Only single family properties that had not changed were considered 












Median Ratio 


Price Related Differential 


Coefficient of Dispersion 


% Improved 


EV Effect 


Zone 


Count 


Percent 


AdjR2 


Start 


Projected 


Final 


Start 


Projected 


Final 


Start 


Projected 


Final 






A 


700 


3.5% 


89.7% 


.310 


.967 


.968 


1,102 


1.024 


1.030 


.328 


.173 


.168 


48.6% 


2.7% 


B 


505 


2.5% 


80.2% 


.485 


.972 


.972 


1.116 


1.059 


1.059 


.287 


.203 


.203 


29.3% 


0.0% 


C 


4,227 


21.0% 


86.3% 


.406, 


.992 


.988 


.992 


1.013 


1,016 


.110 


.088 


.090 


18.6% 


-2.4% 


D 


2,003 


10.0% 


78.2% 


.416 


.999 


.999 


1.004 


1.011 


1.011 


.095 


.079 


.079 


17.0% 


-0.1% 


E 


1,968 


9.8% 


82.0% 


.402 


.994 


.994 


1.011 


1.014 


1.014 


.116 


.093 


.093 


19.8% 


0.1% 


F 


2,175 


10.8% 


82.0% 


.323 


.977 


.977 


.931 


1.051 


1,051 


.412 


.197 


.197 


52.2% 


0.2% 


G 


507 


2.5% 


85.9% 


.259 


1.005 


1.005 


.993 


1.030 


1.030 


.621 


.169 


.169 


72.8% 


0.2% 


H 


502 


2.5% 


78,9% 


.373 


1.055 


1.087 


1.130 


1.108 


1.112 


.567 


.257 


.240 


57.6% 


6.4% 


J 


1,841 


9.2% 


86.0% 


.222 


1.000 


1.000 


.918 


1.045 


1.044 


.383 


.169 


.169 


55.9% 


0.1% 


K 


1,151 


5.7% 


90.5% 


.298 


1.030 


1.030 


.855 


1.050 


1.050 


.586 


.192 


.192 


67.3% 


0.0% 


L 


253 


1.3% 


80.5% 


.376 


.981 


.981 


1.005 


1.031 


1.031 


.255 


,145 


.144 


43.5% 


0.6% 


M 


1,750 


8.7% 


90.4% 


.353 


.967 


.915 


.996 


1.049 


1.075 


.244 


.192 


.199 


18.4% 


-4.0% 


N 


1,107 


5.5% 


82.6% 


.388 


.998 


1.000 


.956 


1.021 


1.016 


.264 


.102 


.097 


63.0% 


4.3% 


P 


1,406 


7.0% 


87.9% 


.329 


1.004 


.999 


.916 


1.031 


1.027 


.391 


.143 


.118 


69.8% 


17.0% 


Overall 


20,095 


100.0% 


84.4% 


.380 


.993 


.990 


.970 


1.031 


1.037 


.275 


.140 


.139 


49.5% 


0.6% 



Condos by Zone 

Using the correct source for sales data and scrubbing the file to remove invalid sales, this is the 
result of the AVI ratio study for single condos 

Performance was measured at 3 points in time - at the start of the project; after the model 
projections; and after the Evaluation staff had reviewed the projections and made corrections 



Actual Value Initiative - Ratio Statistics by Zone for Condos 




Comp 


aring the AVI value of properties that sold against time adjusted prices. Only properties that had not changed were considered 










Median 


Price Related Differential 


Coefficient of Dispersion 


% Improved 


EV Effect 


Zone 


Count 


Percent 


AdjR2 


Start 


Projected 


Final 


Start 


Projected 


Final 


Start 


Projected 


Final 






A 


107 


1.4% 


93.2% 


.219 


.689 


.747 


1.O03 


1.009 


1.024 


.624 


.135 


.140 


77.6% 


-3.6% 


B 


1 


0,0% 


93.2% 


.280 


.912 


.912 


1.000 


1.000 


1,000 


.000 


.000 


.000 






C 


1154 


15.3% 


91.4% 


.474 


.797 


.778 


.970 


1.017 


1,013 


.221 


.120 


.128 


42.3% 


-6.8% 


D 


17 


0.2% 


91.4% 


.429 


.783 


.783 


1.025 


1.046 


1,046 


.105 


.168 


.168 


-59.7% 


0.0% 


F 


161 


2.1% 


93.2% 


.404 


.814 


.835 


.930 


1.037 


1.008 


.545 


.125 


.132 


75.7% 


-5.6% 


G 


32 


0.4% 


93.2% 


123 


,710 


.868 


.921 


1.047 


.992 


.494 


.175 


.112 


77.3% 


35.9% 


H 


118 


1.6% 


93.2% 


.689 


.788 


.726 


1.012 


1.017 


1.017 


.109 


.100 


.097 


10.4% 


2.5% 


J 


30S 


4.0% 


88.2% 


.310 


.799 


.809 


.953 


1,048 


1.024 


.446 


.140 


.113 


74.7% 


19.6% 


K 


755 


10.0% 


88.2% 


.631 


.796 


.793 


.930 


.995 


.995 


.262 


.141 


.125 


52.3% 


11.4% 


M 


193 


2.6% 


94.5% 


.384 


.857 


.787 


1.007 


.975 


.985 


.118 


.107 


.143 


-21.7% 


-33.6% 


N 


280; 


3.7% 


94.5% 


.470 


.926 


.839 


.932 


1.019 


1.024 


.352 


.094 


.102 


71.1% 


-7.8% 


P 


4418 


58.6% 


88.4% 


.318 


.827 


.830 


.963 


1.047 


1.007 


,450 


.136 


.132 


70.7% 


3.0% 


Overall 


7541 


100.0% 


91.9% 


.37S 


.821 


.814 


.986 


1.035 


1.001 


.437 


.135 


.132 


69.9% 


2.6% 



Accuracy Assessment 



The standard ratio study gives a lot of information regarding the quality of assessments, but does not 
address the overall levels of accuracy of the projected values. OPA runs an analysis of the accuracy of 
the projections. 

We measure the difference between the time adjusted sale price (TASP) and the projected value, and 
express that difference as a percentage. This analysis answers the questions 'What percentage of our 
estimates fell within a desired degree of 'closeness' to time adjusted sale prices?' and 'What is the 
degree of accuracy at percentile intervals?' 

Performance was measured at 2 points in time - at the start of the project and after the Evaluation 
staff had reviewed the projections and made corrections 



Where we 'missed', we could miss either low or high. In a perfectly distributed sample, we would 
expect the projected value to be high about as often as it is low. The accuracy assessment also 
reports the percentage of cases where the projections appear to be high. 



Example: 

Time Adjusted Sale Price = $202000 

AVI Projected Value = $195,000 

Difference = -$7,000 

Ratio ($195,000/ $ 202000) = .965 

Percent difference ($ 7,000 / $202,000 ) = 3.46% 



This case would be among the 51.4 % of single 
family residences where AVI projected value was 
within 10% of the TASP. It would also be ranked in 
the 20th percentile of cases where the value fell 
within 3.5% of the sale price 



Accuracy Summary 







lable 1 shows the percentage of AVI values that fall within x% of the Time Adjusted Sales Price "or example, 2014 Market Values for 5%A% of Single Family properties that sold were Within 10% of the Tim» 
Adjusted Sales Pries 




High" measures the percentage of cases where the Market Value to Time Adjusted Sales Price ratio (s between 1.03 and the highest ratio for that group. For example, in the 10% group for single family, it 
represents the percentage of cases that fall between 1.03 and 1.10. It illustrates our intention o x 'missing low rather than missing high' 




jnly thoss cases where properties that were substantially the same when valued as they were a t the time of sale were considered 








1 1 II 1 1 1 1 1 " 










Table 1 - Percentage of Accounts within a Desired Degree of Accuracy 










Single Family 


Multi-Family 


Candos 


Atjartments 


Degree of Accuracy 


Before AVI 


AVI 


High 


Before AVI 


AVI 


High 


Before AVI 


AVI 


High 


Before AVI 


AVI 


High 


10% (Ratio of .9 to 1.1) 


0.4 


51.4 


23.0% 


0.2 


49.7 


25.4% 


1,5 


22 ,3 


16.2% 


LA 


29.4 


24% 


1554 (Ratio of .35 to 1.15) 


0.5 


SS.S 


31.4% 


0.5 


66.2 


27.5% 


2.8 


37.1 


14,054 


1.4 


44.5 


20% 


20% (Ratio of .8 to 1,21 


1.4 


78.5 


32.8% 


0.7 


76.4 


28.5% 


5.1 


52,7 


11.4% 


3.3 


63 


23% 


25% (Ratio of .75 to 1.2SI 




SB 


33.6% 


1 


B3.9 


29,3% 


8.5 


69.3 


10.0% 


3.3 


74.4 


24% 


50% {Ratio of .5 to 1.51 


11 


95.9 


36.0% 


1D.4 


96.6 


30.5% 


31.7 


100 


7,4% 


26.5 


96.2 


2S% 
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Commercial* 


In 


dustrial* 










Degree of Accuracy 


Before AVI 


AVI 


High 


Before AVI 


AVI 


High 


Before AVI 


av; 


High 








10% (Ratio of .9 to 1.1) 


5 


18.S 


32% 


6.2 


19.1 


20 A% 


10.4 


14.1 


21.3% 








1594 (Ratio of .35 to 1.15) 


5.5 


26.6 


39%. 


S.4 


24.2 


23.1% 


14.2 


20 


25.0% 








20?S (Ratio of .8 to 1.2] 


5.S 


33 


46% 


9 


32.6 


25.8% 


18.5 


27.4 


29.6% 








2S% | Ratio of 75 to 1.25| 


a.s 


40,6 


50% 


10.7 


39.9 


26.S% 


25.9 


31.9 


30.1% 








50% (Ratio of ,5 to 1,5| 


20.5 


60.9 


90% 


33.1 


69. J 


29.2% 


54.S 


52.6 


29.7% 




























































Table 2 - Degree of Accuracy at Xth Percentile 












In each percentile represented below, this table shows the percentage difference between the 2014 Market Value and the Time Adjusted Sales Price for each 


jrctperty 


Vpe, 








Example- In 20% of the cases the 20 


14 Market Va 


ue was within 3.5% of th 


1 sales price for Single Family Properties. 


































Percentile 


5ingle Family 


Mufti- Family 


Con d os 


u.pa rtmer-s 


Mixed Use* 


Commercial* 


ndastrbt* 












10th 




i.e% 


4.5S 


3% 


6.7% 


33% 


5.4% 












20 th 


3.5U 


3.8% 


0.9* 


7% 


12.9% 


12.7% 


14,9% 












30th 


5.4X 


S.7% 


12. 7% 


10% 


19.6% 


18.4% 


22.4% 












40th 


7,1% 


7 TV, 


1S.OK 


14% 


26.9% 


26.2% 


29.8% 












50th 


9i6H 


to..],;; 


19. IS 


17% 


37.7% 


31.7% 


42.2% 












50th 


US% 


ll.SU 


22.056 


20'.- 


50.7% 


39.8% 


55.2% 












7Dth 


15.6% 


lfi.B% 


25.3% 


24% 


6S.2% 


53.6% 


70.1% 












30th 


10.i% 


22.3% 


ll.'iri 


2S% 


94.3% 


36.9% 


91.9% 












90th 


1S.2% 


31.6K 


34, 2% 


38% 


143.4% 


221.2% 


197.6% 






































* Sales for this class of property may not be the best indicator of value. Cost Approach, Income Approach, or a Hybrid sales approach may be more reliable 













Measuring the Accuracy of 

AVI Values 



A successful Revaluation 




A visual representation of the change in assessment uniformity 
that was achieved through the 2014 revaluation. 



From Start to Finish 
CHANGE IN RATIO PERFORMANCE 
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Challenges of AVI 



Philadelphia is a very large, old city with a wide variety of property types and a large 
number of geographic market areas 

We faced many challenges in planning and executing the AVI revaluation: 

> Legacy of less than optimal business practices that did not promote uniform values 
or levels of assessment between various classes of property or within a given class 
of property 

> Serious deficiencies in the data files that effected the potential accuracy of any 
mass appraisal effort 

> Outdated computer hardware 

> Critical levels of understaffing in all units and divisions 

> Lack of consistent standards for Sales Validation and for data gathered in the field 

> Lack of integrated CAMA software to support a project of this scope and magnitude 

> The market experienced both a large rise and a significant drop since the last mass 
appraisal 

Any of these issues could have caused the failure of the entire project 
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Setting Realistic Goals 



> We believed it was critical to set realistic goals and performance standards for 
the Mass Appraisal effort. 

> We wanted to inspect all properties and standardize the way that data was 
collected 

> We wanted to validate six years of sales for all property types and capture the 
characteristics at time of sale 

> We wanted to review the Geographic Market Area definitions , and make 
changes or revisions to optimize performance 

> We wanted to use regression models to project values wherever we thought the data 
would yield good results 

> We wanted all classes of property to have a median level of assessment of between 
95% and 102% 

> We wanted a citywide Coefficient of Dispersion of less than 20% 

> We wanted a citywide Price Related Differential of between .98 and 1.05 

> We wanted accuracy within 10% 50% of the time, and within 20% 80% of the time 
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Ratio Studies 



Performance is usually measured by comparing predicted market values 
against the actual, adjusted or time adjusted prices of the properties in the 
inventory that have sold within the analysis period. The relationship 
between market value and sale price is commonly expressed as a 
percentage and referred to as 'the ratio'. Ratio studies that are run against 
the sales used in the model are part of the model calibration process. A 
holdout sample is a file of validated transactions that were not used in the 
valuation process, including new sales that transacted between the date of 
appraisal and the date of the study. Ratio studies are also run against the 
holdout sample. 



Measures of Performance 



The assessment industry recognizes some standard measures of 
performance 

Level of Assessment (Median Ratio) - What is the typical relationship 
between Market Value and Sale Price? 

Horizontal Equity (Coefficient of Dispersion ) - What is the average 
absolute percentage difference between the Market Value and Sale Price? 

Vertical Equity (Price Related Differential) - Does the level of assessment 
remain the same as prices increase? Are we valuing low priced and high 
priced properties at similar levels of assessment? 

Degree of accuracy - What percentage of the time are the values with a 
desired 'closeness' to sales prices? 

Reliability of the model (Adjusted R2)- What percentage of all observed 
variance in sale price is explained by the model? 



Exceeding Expectations 

Summarizing performance results for single family properties, we met or 
exceeded our 1 st year goals in this revaluation 



Measure 

Median Ratio 
Price Related Differential 
Coefficient of Dispersion 
Model Reliability 
Accuracy at 50 th percentile 
Accuracy at 80 th percentile 



1st Year Goal 2nd Year Goal 2014 Result 



.95-1.02 



.98-1.05 



.20 
.80 



.95-1.02 
.98-1.03 

.15 
.85 



.99 
1.037 

.139 
.844 



Within 10% Within 7.5% Within 9.6% 
Within 20% Within 15% Within 20.6% 



A Valid Ratio Study 

In a Ratio study, the analyst should be trying to isolate variance in the market 
value to sales price relationship that is attributable to errors in the estimation 
processes and procedures. Estimation errors that are attributable to flaws in the 
data should be minimized or eliminated. This requires both an understanding of 
the data as well as a high degree of skill in managing the data files. 

The Sales File 

• Must include all transactions for the analysis period 

• Invalid sales should be removed from consideration 

• Truncation of up to 10% is allowable to remove sales that are not good 
indicators of value, although we chose to use a 2% truncation 

• Should use Time Adjusted Sale Price 

The Value File 

• Must include values for all properties in the study 

There are many mistakes that can be made in building and filtering the 
sales file that will lead to erroneous conclusions, therefore only a 
qualified analyst should be engaged. 



Improving Performance 



Based on our analyses and extensive interviews with our staff and expert 
consultants, we have plans to implement enhancements and refinements to 
our business practices and valuation processes that will significantly improve 
the performance of our next revaluation. Among these are: 

• Data quality improvement 

• Better training for the staff 

• Expanding use of income data for non-residential properties 

• Improving the software environment 

• Focusing efforts in areas that need the most improvement 

• More accurate land models 

• Regression model maturity 

• Use of multiple methods of estimating value and reconciling results 

• Expanding use of expert consultants 



